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Abstract 

Various methods for lifted probabilistic inference have been proposed, but our 
understanding of these methods and the relationships between them is still lim- 
ited, compared to their propositi onal counterparts. The only existing theoretical 
characterization of lifting is for weighted first-order model counting (WFOMC), 
which was shown to be complete domain-lifted for the class of 2-logvar models. 
This paper makes two contributions to lifted variable elimination (LVE). First, we 
introduce a novel inference operator called group inversion. Second, we prove that 
LVE augmented with this operator is complete in the same sense as WFOMC. 



1 Introduction 

Probabilistic logical models combine graphical models with elements of first-order logic to com- 
pactly model uncertainty in structured domains (social networks, citation graphs, etc.) 13] |9). These 
domains can involve a large number of objects, making efficient inference a major challenge. Lifted 
probabilistic inference methods address this problem by exploiting symmetries present in the struc- 
ture of the model |[I]|5][iS[IT][I2[E3][ll[l5]Q^ The basic principle is to identify "inter- 
changeable" groups of objects and perform an inference operation once per group instead of once per 
individual in the group. Researchers have proposed "lifted" versions of many standard propositional 
inference algorithms, including variable elimination J5J Q4] Q3), belief propagation HV2\ [T8l \19\ , 
recursive conditioning 1161 . weighted model counting 1 10 1 and knowledge compilation l8l l2"Tl . 

Despite the progress made, we have far less insight into lifted inference methods than into their 
propositional counterparts. Only recently has a definition been proposed for lifted inference. 
Domain-lifted inference requires the time-complexity of inference to be at most polynomial in the 
domain size (number of objects) of the model J5). In contrast, standard propositional inference is 
typically exponential in the domain size in probabilistic logical models. Given this definition, it is 
possible to theoretically characterize, in the form of completeness results, which classes of mod- 
els always permit lifted inference. Weighted first-order model counting (WFOMC) is the first lifted 
algorithm shown to be complete for a non-trivial model class. Van den Broeck |8| showed that 
WFOMC is domain-lifted complete for 2-logvar models. These models can express many important 
regularities, that commonly occur in real-world problems such as (anti-)homophily and symmetry. 

We continue this line of theoretical work by analyzing lifted variable elimination (LVE) lfTll4l [T4l[T5l . 
We advance the state of the art in LVE in two ways. First, we propose a novel LVE operator called 
group inversion that generalizes the basic inversion operator. This operator expands the situations 
where LVE exploits the symmetry between interchangeable parts of the model, and hence increases 
the number of operations that can be done efficiently, via lifting. Second is our main result: we 
prove that augmenting LVE with group inversion makes it complete for the same class of models as 
WFOMC, namely 2-logvar models. This theoretical result establishes a connection between LVE 
and WFOMC, and also shows the importance of the the new operator. 
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The paper is structured as follows. In Section|2]and[3]we give the necessary background on represen- 
tation and algorithms. In Section|4]and|5]we introduce and extend the new group inversion operator. 
We prove completeness of LVE with the new operator in Section[6| and conclude in Section[7] 

2 Representation 

Probabilistic logical models combine graphical models with elements of first-order logic. Many rep- 
resentation languages exist for such models 0. Like earlier work on LVE lfT5l l4l fT4l . we represent 
the model with parametric factors . This formalism can compactly represent undirected probabilistic 
models on large numbers of objects. We now introduce the necessary terminology. 

We use the term "variable" in both the logical and probabilistic sense. We use logvar for logical 
variables and randvar for random variables. We write variables in upper- and values in lowercase. 

We consider factorized probabilistic models. A factor f — 4>/(Af), where Af — (A\, . . . , A n ) are 
randvars and <fif is a potential function, maps configurations of Af to a real-number. An undirected 
model is a set of factors F over randvars A = U/gf Af and represents the following probability 
distribution: Vf(A) = ^ YlfeF ^/ (Af)> w ' tri % a normalization constant. 

Our representation compactly defines a set of factors, using concepts of first-order logic. A constant 
represents an object in our universe. A term is either a constant or a logvar. A predicate P has an 
arity n and a finite range (range(P)); it maps n-tuples of objects (constants) to the range. An atom 
is of the form P(t\,t2, ■ ■ ■ , t n ), where the ti are terms. A ground atom is an atom P(c%, . . . , c„) 
where the Cj are constants. The range of such a ground atom is range(P). Each ground atom 
represents a randvar (e.g. BloodType(joe)). Note that the range of predicates, and hence randvars, 
is not limited to {true, false} as in logic (e.g., range(BloodType(joe)) — {a, b, ab, o}). 

Each logvar X has a finite domain, D(X), which is a set of constants {x±, . . . , x n }. A substitution, 
9 = {X\ — > ti, . . . , X n — > t n }, is a mapping of logvars to terms. A grounding substitution maps 
all logvars to constants. Applying 6 to a, denoted aO, replaces each occurrence of X{ in a with 4j. 

A constraint Cx on a set of logvars X = {Xi, . . . , X n } is a conjunction of inequalities of the 
form Xi ^ t where t is a constant in D(Xi) or a logvar in X (the conditions Xi € D(Xi) are left 
implicit). We write C instead of Cx when X is apparent from the context. By gr(X|Cx) we denote 
the set of ground substitutions to X that are consistent with Cx- 

A parametrized randvar (PRV) is a constrained atom of the form P(X)|C, where P(X) is an atom 
and C is a constraint on X. A PRV P(X)|C represents a set of ground atoms, and hence a set of 
randvars, {P(X.)9\8 e gr(X.\C)}. Given a PRV V, RV(V) denotes the set of randvars it represents. 

Example. The PRV V = Smokes(X)\X ^ x\, with D{X) = {x\, . . . ,x n }, represents n — 1 
randvars {Smokes(x2), ■ ■ ■ Sraokes(x n )}. □ 

A parametric factor or parfactor is of the form VL : C.<fi(A), with L a set of logvars, C a constraint 
on L, A — (Ai)™ =1 a sequence of atoms parametrized with L, and 4> a potential function on A. 
The set of logvars occurring in A is denoted logvar(A), and we have logvar(A) C L. When 
logvar(A) = L, we write the parfactor as 4>(A)\C. A factor cj)(A') is called a grounding of a 
parfactor <f>(A)\C if A' can be obtained by instantiating L according to a grounding substitution 
8 6 gr{h\C). The set of all groundings of a parfactor g is denoted gr(g). 

Example. Parfactor g = <fi(Smokes(X), Asthma(X)) represents the set of n ground factors 

gr{g) = {4>(Smokes(xi), Asthma(xi)), . . . , 4>(Smokes(x n ), Asthma(x n ))}. □ 

When talking about a model below, we mean a set of parfactors. In essence, a set of parfactors G 
is a compact way of defining a set of factors F — {f\f £ gr(g) Age G}. The corresponding 
probability distribution is Vg{A) = \ EI/Gf ^/ (Af )■ 

3 (Lifted) Variable Elimination 

The state of art in lifted variable elimination (LVE) is the result of various complementary efforts |[T5l 
[El [Hill. This section reviews the most recent algorithm for LVE, namely C-FOVE fl4l . 
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Variable elimination calculates the marginal distribution of some variable by eliminating randvars 
in a specific order from the model until reaching the desired marginal ifTTl . To eliminate a single 
randvar V, it first multiplies all factors containing V into a single factor and then sums out V from 
that single factor. LVE does this on a lifted level by eliminating parametrized randvars (i.e., whole 
groups of randvars) from parfactors (i.e., groups of factors). The outer loop of LVE is as follows. 



Inputs: G: a model; Q: the query randvar. 
while G contains other randvars than Q: 

if a PRV V can be eliminated by lifted sum-out 
G <— eliminate V in G by lifted sum-out 

else apply an enabling operator on parfactors in G 
end while 
return G 



As this shows, LVE works by applying a set of lifted operators. We now discuss the most basic 
operators. Beside these, LVE has conversion operators, which we discuss in Section|5] 

Lifted Sum-out. This operator sums-out a PRV, and hence all the randvars represented by that PRV, 
from the model. Lifted sum-out is applicable only under a precondition (each randvar represented 
by the PRV appears in exactly one grounding of exactly one parfactor in the model). The goal of 
all other operators is to manipulate the parfactors into a form that satisfies this precondition. In this 
sense, all operators except lifted sum-out can be seen as enabling operators. 

Lifted Multiplication. This operator performs the equivalent of many factor multiplications in a 
single lifted operation. It prepares the model for sum-out by replacing all the parfactors that share a 
particular PRV by a single equivalent product parfactor in the model. 

Splitting and Shattering. These operators rewrite the model such that a pair of atoms or formulas 
represent either identical or disjoint groups of randvars. 

4 A New Operator: Group Inversion 

We now introduce a new lifted operator called group inversion. This operator generalizes the existing 
inversion operator of FOVE ||5] [T3) and is inspired by the concept of disconnected groundings in 
lifted recursive conditioning [16]. We first review the existing inversion operator, and then define 
group inversion. The motivation behind adding this new operator to LVE is that it makes LVE 
complete for important classes of models, see Section|6] 

4.1 Inversion 

Lifted sum-out eliminates a PRV, i.e., a whole group of randvars, in a single operation. An important 
principle that it relies on is inversion |[T5l 0] . 

Inversion consists of turning a sum of products into a product of sums. Consider the sum of products 
J2i J2j * ' J- If tne ran g e of j does not depend on i, it can be rewritten as (J^ . *)> which is a 

product of sums. More generally, given n variables x\, . . . , x n , with independent ranges, we have 

££-£n/^)=n£/^)- 

x\ X2 x n i i Xi 

Furthermore, if all Xi have the same the range, this equals (Y2 Xl f{ x i)) n - Th at * s ' me summation 
can be performed for only one representative x\ and the result used for all Xi. 

Exactly the same principle can be applied in lifted inference for summing out randvars. Suppose 
we need to sum out RV(F(Xi,X 2 )) from the parfactor g = <j>{F{X\, X 2 ), P(Xi, X 2 )). For each 
instantiation (xi, x 2 ) of (X ll X 2 ), F{x\, x 2 ) has the same range, hence applying inversion yields 

£ £ - £ n^=n( £ 90) a) 

F(xi,xi) F(xi,x a ) F{x„,x n )9eB 6eB F(X U X 2 )6 

with = gr(Xi, X 2 ). This shows that we can perform the sum-out operations independently for 
each F(Xi, X 2 )9, and multiply the results. Furthermore, since all the factors g6 are groundings 
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of the same parfactor and have the same potential <f>, the result of summing out their first argument 
F(X±, X 2 )9 is also the same potential, denoted <fi'. It thus suffices to only perform one instance of 
these sum-out operations and rewrite Expression[TJas 

JJ( E <i>{F(X ll X 2 )6,P{X 1 ,X 2 )e)) = Y{( ( t ) '{P(X 1 ,X 2 )e))=gr(g') 

eee F{x 1 ,x 2 )e eee 

where g' = (j>'(P(X\, X 2 )). This is what lifted sum-out by inversion does: it directly computes 
parfactor g' from g — <p(F(X\, X 2 ), P(X\, X 2 )) by summing out F(Xi,X 2 ) from g in a single 
operation. This single lifted operation replaces |0| sum-out operations on the ground level. 

4.2 Group Inversion: Principle 

Inversion only works when the summations are independent. Our first contribution is based on the 
following observation. When we cannot apply inversion because of dependencies between factors, 
we can still partition the factors (and the summations) into groups such that dependencies exist only 
among factors within a group, but not between groups. Furthermore, we can compute the result for 
one group and use it for all groups, provided that these groups are isomorphic, i.e., that there exists 
a one-to-one-mapping of the randvars from one group to the others such that exactly the same sum 
of products is obtained. We call this group inversion. 

Consider the parfactor g = <fi(F(Xi, X 2 ),F(X 2 ,Xi))\Xi ^ X 2 and assume we want to sum out 
the randvars RV(F(Xi, X 2 )\Xi ^ X 2 ). If we focus on the part of the computation related to one 
particular instantiation {x\, x 2 ), the sum looks as follows. 

... J2 ■■■ E ...{4>(F(x 1 ,x 2 ),F(x 2 ,x 1 ))- ( f>(F(x 2 ,x 1 ),F{x 1 ,x 2 )) •(...) 

F(x 1 ,x 2 ) F(x 2 ,x 1 ) 

The product contains two factors over the considered pair of randvars F(x\, x 2 ) and F(x 2 , x\). The 
product of these two cannot be moved out of the summation over either of the two randvars. Still, 
the two summations are independent of all other factors, and can be isolated from the rest of the 
computation. The same can be done for each pair of instantiations (xi,Xj) of (Xi, X 2 ). This means 
that summing out RV(F(X 1 , X 2 )\Xi ^ X 2 ) fromgcanbe done using group inversion: 

E E ••• E (11*%) = II t E E *v^«) 

f(x 1 ,x 2 ) f(x!,x 3 ) f(x„_i,x„) e„ee !■.< .</ :• <•> F(x 1 ,x 2 )e, j F(x 1 .x 2 )e ]i 

where dij is a grounding substitution {X\ — > Xi, X 2 —> Xj} in 9 = gr(X\, X 2 \X\ 7^ X 2 ). Lifting 
is now possible again because for all distinct pairs of substitutions (6ij , Oji) in Q, the pairs of factors 
(gOij,gdji) share the same potential cf>. As such, the multiplicands of each pair also have the same 
potential cf>', and summing out their arguments results in the same potential <fi". Hence, it suffices to 
perform only one (lifted) instance of these operations as follows 

II ( E E 4>'(F(X 1 ,X 2 )e ij ,F(X 1 ,X 2 )9 ji )) 

{e»j,0jx}ee F{x 1 ,x 2 )e ij f(Xi,x 2 )0j« 

where g' is the parfactor VXi, X 2 : X\ ^ X 2 .<fi"Q with <j>" a potential function with no arguments 
(i.e., a constant, because both arguments have been summed-out). This is what group inversion does. 

Group inversion partitions the set of factors (and randvars) into independent and isomorphic groups. 
An important question is what such a partitioning looks like. Figure [T] shows this for the above 
example. In general, let us call two factors directly linked if they share a randvar, and let linked 
be the transitive closure of this relation. Factors that are linked end up in the same group. Some- 
times this yields useful partitionings, sometimes not. As a 'negative' example, consider a parfac- 
tor 4>(P(Xi),P(X 2 )). Any two ground factors <j)(P{x i ) 1 P{xj)) and 4*(P{x k ), P{%i)) are linked 
(since both are directly linked to <j)(P(xj), P(xk)))- Hence the only option is the trivial partition in 
which all ground factors are in a single, large group, which is not practically useful. As a 'positive' 
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Figure 1: Group inversion on pairs of randvars. Circles represent randvars, squares represent factors. 
Dashed boxes indicate the partitioning into groups. 



example, consider the case where each atom uses all the logvars in the parfactor, as in the earlier 
example <fr(F(X±, X 2 ), F(X2, Xi))\Xi ^ A 2 . In such cases, we can always partition the randvars 
into groups such that the size of a group is independent of the domain size. The reason is that in 
such cases, the arguments of the atoms in a linked group are necessarily permutations of each other. 
Hence the size of a linked group can be no larger than the number of possible permutations, which 
is independent of the domain size. We use this property in our group inversion operator. 

4.3 The Group Inversion Operator 

In- and output. Group inversion takes a parfactor g = <p(A)\C and a set of atoms {Ai, . . . , A n } C 
A as input. It returns a new parfactor that is the result of summing out {A 1} . . . , A n } from g. 

Preconditions. Group inversion is applicable when (i) for all i, j: RV(Ai\C) = RV(Aj\C), (ii) 
each Ai has all the logvars L in the parfactor, (iii) for each pair of logvars Xi , X,- G L there is an 
inequality constraint X t ^ Xj in C, and (iv) for each PRV V outside g: RV(Ai\C) H RV(V) = 0. 
The key observation is that, in such a parfactor, due to conditions (i) and (ii), for each i 7^ j, randvar 
Ai is a permutation of Aj. That is, Ay(vlj) = Aj, where Ay is a permutation of the logvars and 
X(Ai) represents the result of applying A on the arguments of Ai. 

Operator. When the preconditions hold, group inversion applies the following four steps. We 
further explain these steps below. 

1 . Partition. Find the set A of permutations Ay such that Ay (Ai ) = Aj. Then find the closure 
[A] of A, i.e., the minimal set [A] of permutations such that AC [A], and [A] is closed under 
composition. 

2. Multiply to compute the parfactor = <j)'(A')\C as the product IlAe[A] Sa> where g\ = 
0(X(A))[C. 1 " 

3. Sum-out to compute g' = cf)"(A")\C ~ X^ [A] 9[A]> where the summed out atoms A[a] = 
{A[ . . .,A' m ] are such that : RV(A t \C) = RV{A'AC). 

4. Scale. Return g" = 0" '(A") 1/m \C ', with m = |[A]| (or equivalently: m = \A [A ]\)- 

Stepl (partition). The goal here is to find, on the lifted level, the set of factors and randvars that 
need to be put (and summed-out) in the same group. In gr(g), each pair of factors (g9, gO') that are 
directly linked can be derived from each other by a permutation of constants. Concretely, if randvar 
Afi in g9 is the same as AjO' in g9', then we have 6 — A^ All factors that are directly linked 
to a factor g6 are thus in the set {g0'\9' = X(0), A € A}. In Step 1, we find the set of permutations 
A that convert directly linked factors to each other. Since directly linked factors are derived from 
each other by a permutation in A, each factor linked to g9 can be written as gX(6), where A is a 
composition of permutations in A. We can thus find all such factors by computing the closure of 
A under the operation of composition, this is also called the minimal permutation group. That is, 
the closure is the minimal set [A] D A, such that VAi, A 2 G [A] : Ai.A 2 G [A]. Note that all the 
permutations are computed on the lifted level, i.e., as permutations of the logvars. 

Example. Consider finding the closure [A] for a parfactor with atoms A\ — F(W 1 X,Y, Z) and 
A2 = F(Z, W, X, Y). Note that a right-shift of Ai's logvars yields A^. Let us denote a permutation 
A = {W -> W, . . . Z — > Z 1 } as A = (W, . . . , Z'). For atoms {A 1 ,A 2 }, A includes the three 
permutations An = A 22 = (W,X,Y,Z), X 12 = (Z,W,X,Y) and A 2i = (X,Y,Z,W) (respec- 
tively identity, right- and left-shift). A is not closed under composition because, for example, the 



'Note that for any atom Ai, and a permutation A of its logvars, AiX(8) — \(Ai)8. 



composition A' = A12.A12 = (Y, Z, W, X) is not in A. Adding A' to A, however, yields a closed set 
of permutations, as no other possible composition results in a permutation out of this set. As such, 
we have [A] = A U {A'}. Note that [A] includes only 4 of all the possible 4! = 24 permutations. □ 

Step 2 (multiply). Next we apply lifted multiplication to compute a parfactor = I1a6[A] 5a > 
where g\ = <f>(X(A))\C. This parfactor has the same form as the product of all the factors in a 
representative group (for a given order of multiplications). 

Step 3 (sum-out). Next we perform lifted sum-out, which sums-out the set of atoms {A(^4i)|A <E 
[A]} from parfactor gui and yields parfactor g' . At this point, all randvars RV(Ai\C) have been 
eliminated from the model, which is the goal of the entire group inversion procedure. 

Step 4 (scale). For correctness, we still need to scale g' , as it represents multiple equivalent factors. 
Since g' has the same constraint as g and hence the same set of all possible grounding substitutions 
0, grig') represents |[A]| equivalent factors instead of one factor, for each group of the factors in 
<?r(g)P| Hence, we replace the potential 0" of g' with <\> 1 /H A 1I to preserve the distribution. 

Remarks. Since group inversion includes parfactor multiplication, beside removing the summed- 
out atoms, it can add to the resulting parfactor additional arguments that result from permutation of 
logvars in its original arguments. The following example depicts this. 

Example. Consider applying group inversion to eliminate the set of randvars RV(F(X, Y)) from 
g = <f>(S(X),F(X,Y),F(Y,X),A(Y))\X ^ Y. In Step 1, we find the permutation group [A] 
for Ai = F{X,Y) and A 2 = F(Y,X). The group [A] = {A, A'} consists of the identity per- 
mutation A = {X -> X, Y -> Y}, and the permutation A' = {X -t Y, Y -t X}. In Step 
2, we multiply g x = g and g x , = (f>(S(Y), F(Y, X),F(X, Y), A(X))\X ^ Y, to compute 
g [A] = cj)'(S(X),A(X),F(X,Y),F(Y,X),S(Y),A(Y))\X ^ Y. In Steps 3 and 4, we respec- 
tively sum-out atoms F(X, Y), F(Y, X) from gui and scale the resulting potential, to compute the 

parfactor g" = (f>"(S(X),A(X), S(Y), A{Y)f/ 2 \X ^ Y. Note that g" includes atoms A(X) and 
S(Y), which were not present in the original parfactor g. □ 

As mentioned before, group inversion has inversion as a special case (namely when there is only one 
atom in the parfactor that covers the randvars RV (Aj|C), i.e., when n = 1 in the operator). In this 
case, the closure [A] consists only of the identity permutation. 

Theorem 1 Lifted sum-out with the group inversion operator is equivalent to summing out the rand- 
vars on the ground level. 

The proof is provided in the appendix (it relies on showing that the corresponding ground operations 
are independent and isomorphic). 

5 Extension for Counting 

LVE exploits symmetry among groups of interchangeable and independent objects, as we have seen 
in group inversion. Additionally, LVE also exploits symmetry within groups of interchangeable 
but dependent objects, by counting. Inspired by counting elimination [4|, Milch et al. introduced 
counting formulas and operators that handle them in the C-FOVE algorithm [14]. This was later 
extended by Apsel and Brafman [TJ. These operators play a key role in LVE and in deriving our 
completeness results (Section[6]i . Below we first review counting formulas and the existing operators 
that handle them. Then we extend group inversion with support for summing-out counting formulas. 

Counting formulas. A counting formula is of the form 7 = #x-.c[P{X)\i with C a constraint on 
the counted logvar X £ X (if the only condition is X £ D(X), we do not write out C). Because 
the counted logvar X is already bound by the counting formula, it is excluded from logvar(j). A 
grounded counting formula is a counting formula in which all logvars except the counted logvar 
are replaced by constants. Such a formula represents a counting randvar (CRV). The range of the 

CRV is the set of possible histograms of the form h = {(^, rii)}|™ 1 nse ' P '", that shows for each 



2 This equivalence follows from the fact that each of them can be derived from a same set of factors, but 
with a different order of multiplications in Step 2. 
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ri G range(P) the number of randvars P(. . . , x, . . .) whose state is r,-. The state of the counting 
randvar thus depends deterministically on the state of the randvars RV(P(. . . , X, . . .)\C). 

Example. #y Y)] is a counting formula and #y , Y)] a grounded counting formula that 

counts the number of friends of x\. Having D(Y) = {3/1,2/2,2/3}, the grounded counting formula 
covers the randvars {F(xi, 2/1), F(x\, 2/2), F(xi, 2/3)} and defines a CRV, the value of which is 
determined by the values of these three randvars. For instance, if F(xi, j/i) = true, F{x\, 2/2) = 
false and F(x\, 2/3) = true, the value of the CRV is the histogram {(true, 2), (false, 1)}. 

Counting formulas can be introduced in the model by the following two conversion operators. 

(Just-different) Counting Conversion lTT4l IT1 . This operator introduces counting formulas in the 
model, to exploit the symmetry between a group of interchangeable randvars. By replacing an 
atom such as A(X) with a counting formula #x [A(X)], we compactly represent and manipulate a 
single potential on randvars A(x\), . . . , A(x n ). Intuitively, this conversion achieves the equivalent 
of multiplying groundings of a single parfactor with each other. This operator is applicable on 
a set of logvars that only appear in a single atom or in just-different atoms, that is, pairs of atoms 
P(X±, X), P(X2, X) whose logvars are constrained as X\ 7^ X2, such as atoms A(X\) and A(X2) 
in the parfactor ^(A(X 1 ),A(X 2 ))\X 1 ^ X 2 . 

Joint Conversion |fl~). This operator is an enabling operator for counting conversion. Joint con- 
version on a pair of atoms A(X), B(X) replaces any occurence of A(.) or B(.) with a joint atom 
Jab(-), whose range is the Cartesian product of the range of A and B. Such a conversion enables 
counting conversion, when it results in a model with just-different joint atoms. 

Example. Consider the parfactor 4>(S(X), A(X), S(Y), A(Y))\X 7^ Y. By joint conversion on 
atoms S(.) and A(.), we rewrite this parfactor as 4>'(J S a(X), J S a(X), J sa (Y), J sa (Y))\X ^ Y, 
which is trivially simplified to (\>"(3 SA (X), J S a{Y))\X ^ Y. Note that Jsa(X) and Jsa(Y) are 
just-different atoms. By just-different counting conversion, we rewrite this parfactor into the form 
4ft x[J s a(X)\) . This parfactor is now prepared for application of lifted sum-out. 

Extension of Group Inversion. The above are existing operators. Now that we have counting 
formulas, we also need to support them in our new group inversion operator. Counting formulas, 
like atoms, can be eliminated by lifted sum-out. This can be done by group inversion with a small 
modification of the operator. When {Ax, . . . , A n } is a group of counting formulas of the form 



Ai = # Xl :C> 



in Section 4.3 



P(Xi, L)], the operator eliminates all the formulas by following the same 4 steps as 
with the exception of the sum-out step (Step 3), which becomes: 



j2 ((n NuM (^)) 

(hi,...,h m )£range(A' 1 ,...,A' m ) i=l 

where, for a histogram hi = {(r^, rii)}[ =1 with ^ rii — n, the coefficient NUM(hi) is the multi- 
nomial coefficient p-p^rr representing the number of possible assignments to RV(A' i \C) that yield 
this histogram. This operator generalizes the existing sum-out operation of LVE fl4l . 



6 Completeness Results 

We now show that including our group-inversion operator yields certain completeness results. These 
are the first such results for LVE and the second for exact lifted methods in general (after WFOMC 
(HI). The concrete LVE algorithm considered is C-FOVE [14J extended with group-inversion and 
joint formulas [ 1 ]. We refer to it as C-FOVE+. We use the definitions of Van den Broeck |8 1. 

Definition 1 (Domain-lifted algorithm) A probabilistic inference algorithm is domain-lifted/or a 

model G, query Q and evidence £ iff it runs in polynomial time in |m|, \rik\, with rii the domain 
oflogvarXi G logvar(G, Q,£). 

Definition 2 (Completeness) An algorithm is complete for a class A4 of models if the algorithm is 
domain-lifted for all models G G Ai and all ground queries Q and evidence £. 
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6.1 Main Result: Completeness for 2-logvar models 

We show that C-FOVE+ is a complete domain-lifted algorithm for the subclass of parfactor models 
called 2-logvar models. A parfactor model is 2-logvar if no parfactor has more than 2 different 
logvars, and the model contains no counting formulas. We discuss the importance of this class of 
models in the next section. 

Theorem 2 C-FOVE + is a complete domain-lifted algorithm for 2-logvar models. 

Proof. We show the proof here since it is 'constructive': it shows how C-FOVE+ deals with 2-logvar 
models: first sum-out all 2-logvar atoms (atoms A with \logvar(AY\ = 2), then 1-logvar atoms, then 
0-logvar atoms. We then show that this procedure is domain-liftedFj 

Step 1. We eliminate all 2-logvar atoms in two steps, (a) We multiply the parfactors until there are 
no distinct pairs (Ai, Aj) of 2-logvar atoms in distinct parfactors (gi, gj), such that RV(Ai\Ci) = 
RV(Aj \Cj). The resulting equivalent model M* is a 2-logvar model, since multiplication preserves 
the number of logvars in the product 11141 . (b) We eliminate each 2-logvar atom in M* by group 
inversion. This is possible since all 2-logvar atoms that represent the same randvars are in the same 
parfactor, and have both of the logvars. The result is a 1-logvar model. 

Step 2. We eliminate all 1-logvar atoms in four steps, (a) We (repeatedly) perform joint conversion 
on a pair of atoms Pi (Xx , Ci) and P 2 (X 2 , c 2 ) (of distinct predicates). This replaces them with joint 
atoms J%2(Xi, C12) and J\ 2 (X 2: C12), respectively. When no more such conversions are possible, 
any pair of logvars X\, X 2 that are constrained as X\ ^ X 2 , appear only in pairs of just-different 
atoms (otherwise, a joint conversion is still possible between them), (b) We perform (just-different) 
counting conversion on all the logvars. (c) We multiply all the parfactors into one, which is trivially 
possible since the model contains no free logvars. In the resulting parfactor each argument is either a 
ground atom, or a counting formula of the form 7; = j^Xi \J\...ki (Xi)]. (d) We sum-out the counting 
formulas. The result of this step is a model in which all arguments are ground, i.e., a 0-logvar model. 

Step 3. We eliminate all the remaining (so 0-logvar) non-query randvars. Inference is now per- 
formed at the ground level, i.e., with standard VE. This step concludes the inference process. 

Complexity. All the operations in Steps 1 and 2 run in time polynomial in the domain of the logvars. 
The most expensive step is handling the counting formulas produced in Step 2b. The largest size for 
the range of these formulas is 0(n r ) where r is the largest range size among the (joint) atoms. As 
such the exponent r is independent of the domain size. The complexity of Step 2 is thus polynomial 
in the domain size. Step 3 has worst case complexity 0(m c ), with c the total number of symbols 
appearing in (M, Q,£), and m the largest size of range among the randvars. This complexity 
satisfies the definition of a domain-lifted algorithm (8). As such, C-FOVE+ is a domain-lifted 
algorithm for any 2-logvar model, and hence complete for this class of models. □ 

6.2 Importance of the Result 

Our completeness result furthers our understanding of the relation between LVE and lifted search 
based methods, which is an important problem in the field lfT0ll2Tl[T6ll . Van den Broeck [8] showed 
that his WFOMC algorithm is complete for the class of 2-WFOMC problems. Any such problem 
can be represented as a 2-logvar model, and vice versa (see the appendix). Our completeness result 
for LVE is thus equally strong as that of Van den Broeck for WFOMC. 

The class of 2-logvar models includes many useful and often employed models in statisti- 
cal relational learning. It can model multiple kinds of relations, including: homophily be- 
tween linked entities, e.g., cf>(Property(X), Related(X,Y), Property (Y)); symmetry, e.g., 
<p(Friend(X,Y), Friend(Y, X)); anti- symmetry, e.g., (j)(Smaller(X,Y), Smaller(Y, X)); and 
reflexivity, e.g., <fi(Knows(X, X)). The completeness result shows that for these models, 
LVE can perform inference in time polynomial in the domain size. An example of mod- 
els that fall outside of the 2-logvar class are models containing a transitive relation, e.g. 
<j)(Like(X, Y), Like(Y, Z), Like(X, Z)). For such models, no domain-lifted inference procedure 



3 We assume that the model M is preemptively shattered and in normalized form 11411161 . Any 2-logvar 
model M can be rewritten in poly time as an equivalent 2-logvar model M' that satisfies these conditions. 
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is known. An important direction for future work is the derivation of (positive or negative) com- 
pleteness results for such model classes. 

Next to our main result (Theorem 2), we now present a second result (Theorem 3) that is in line with 
a known result for lifted recursive conditioning fl6l . This result applies to models that restrict the 
number of logvars per atom, while our main result restricts the number of logvars per parf actor. 

Theorem 3 C-FOVE + is a complete domain-lifted algorithm for the class of models in which each 
atom has at most 1 logvar. 

The proof is provided in the appendix (it builds on the proof of Theorem 2). 

7 Conclusion 

We showed how introducing a new inference operator, called group inversion, makes lifted variable 
elimination a complete domain-lifted algorithm for 2-logvar models. A corollary of the complete- 
ness result is that lifted variable elimination and WFOMC are domain-lifted complete for exactly 
the same subclass of models. We believe that future research on the relationships between the var- 
ious lifted inference algorithms will yield valuable theoretical insights, similar to those about the 
propositional inference methods ||2]|7l[6]. 

Appendix 

In this appendix we provide proofs for Theorem 1 and 3, and present a procedure for transforming 
weighted model counting (WMC) models to parfactor models. 

A Proof of Theorem 1 

We prove the theorem by showing that the corresponding ground operations are both independent 
and isomorphic. 

Independence. We require the following definition. 

Given a set of factors F and set of randvars R, we call a subset of factors F' C F mutually closed 
with respect to a group of randvars R' C R, if (i) no factor in F \ F' contains a randvar r' G R', 
(ii) no randvar in R \ R' appears in a factor /' G F', and (iii) each randvar r' G R' appears in some 
factor /' e F'. 

Now, we show that we can form mutually closed sets of randvars and factors in R — RV(Ai\C) and 
F = gr(g) by partitioning them into sets in which all elements are permutations of each other (can 
be derived from one another by a permutation of constants). The set of permutations that defines the 
partitioning is the minimal permutation group [A] . 

Given a set of permutations A on X, we define two substitutions 61,82 to be in the relation iff 
A(#i) = 82 for some A G A. Using this relation we can define a partitioning of a set of substitutions 
9 as ©a, where 9 and 8' are in the same group if and only if 8 ^\ 8'. 

As shown in steps 1 and 2 of the operator, for any two factors g8 and g& that share a randvar from 
the set RV(A i ), we have 8 — X(8'), for some A G [A]. Thus for any 9^ G ®[a]> me set of factors 
Fi — <= ®i} are mutually closed w.r.t. the set of randvars Ri = {Ai8\8 G 9^}. This shows 

that we can divide the problem of summing out RV(Ai) from gr(g) into independent problems of 
summing out each set of randvars Ri from the set of factors F,;. 

Isomorphism. We show that the sum-out problems are also isomorphic, by a mapping between the 
ground substitutions that produce ground factors in each group. 

To show the isomorphism between groups of gr(g), we note that each group is formed from the 
factors {g8\8 G 9;}, where 9^ is a group in 8m, The one-to-one mapping between the factors can 
thus be established by a one-to-one mapping between the constants of the grounding substitutions in 
different groups 9^ and 9^ . This is done by starting from an arbitrary pair of substitutions 8i G 9; 
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and 6j E Qj and mapping the constants that are assigned to the same logvar to each other. It follows 
then that each substitution 6' i E Oj such that \(6i) = B' i is mapped to exactly one substitution 
6j E Qj such that \(6j) = 0'j. As such the set of factors (and the set of randvars) are isomorphic up 
to a renaming of the constants in each group. 

This shows that the sum-out problems in different groups are independent and isomorphic. Hence, it 
is correct to replace them by a single lifted operation, i.e. to solve one instance of the problem for a 
representative group and generalize the result for all, as is performed in lifted sum-out by the group 
inversion operator. 



B Proof of Theorem 3 



Proof sketch. The proof builds on the proof of Theorem 2 (given in the paper). Note that the 
approach used in Steps 2 and 3 of the proof of Theorem 2 is also applicable here. The operations in 
Step 2, which together eliminate the 1-logvar atoms, do not depend on the total number of logvars 
in the parfactors. Using this approach, we can eliminate all the 1-logvar atoms in any model whose 
atoms contain at most one logvar. The resulting model can be solved as in Step 3. As was shown in 
the proof of Theorem 2, the time-complexity of these steps is polynomial in the domain size. The 
inference procedure is thus domain-lifted. □ 



C Transformation from WMC to Parfactor Models 



In this section we introduce a method for transforming any weighted model counting (WMC) 
model ll2Tl l8l [Toll to an equivalent parfactor model lfT4l[T5l l4ll. i.e., a transformation from the repre- 
sentation used by WFOMC to the representation used by LVE. 

A WMC model M = (C, w) consists of a set of constrained clauses C and a weight function w that 
maps each predicate P to a weight w(P). We present a transformation from such a model to an 
equivalent parfactor model. Given any k- WFOMC model (with clauses containing up to k logvars), 
the following transformation method returns an equivalent fc-logvar parfactor model (with parfactors 
containing up to k logvars). 

Consider a WMC model M with the weighting function w and the set of constrained clauses C = 
{{CU, Ci)}f =1 , where CU is a disjunction of literals of the form -P(X) or ^P(X), and Ci is a 
constraint on the logvars. We transform this model to a parfactor model M' consisting of two 
groups of parfactors: 

Weight parfactors First we consider the weight function w. For each predicate P in M we add 
a parfactor 0p(P(X)) to M' , with potential <f>p defined as: 4>p(true) = w(P) and 
4> P {fal S e) = l-w(P). 

Clause parfactors Now we consider the set of constrained clauses C. For each constrained clause 
(Cli,Ci) E C, we add a parfactor <pi(Ai)\Ci to M', where Ai is the set of atoms that 
appear (in negated form) in clause CU, and the potential <fii is defined such that for any 
assignment of values a to Ai'. <fii(&) = 1 if a satisfies CU, and 4>i(a.) — otherwise. 

This transformation maps any WMC model M to a parfactor model M' that defines the same prob- 
ability distribution as AI. The following example illustrates such a transformation. 

Example. Consider the 2-logvar WMC model M consisting of the weight function w, and the 
constrained clause, 

^P(X) \/Q(Y)\X^Y 

Using the above method we derive the equivalent parfactor model M' consisting of the following 
set of parfactors: 

• Two weight parfactors <fip(P(X)) and 4>q{Q(X)), with potentials cf)p and <f>Q defined as 
follows: 



p 


<t>p 


false 
true 


1 - w{P) 
w{P) 



Q 


4>Q 


false 


l-w(Q) 


true 


w(Q) 
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• One clause parfactor <p(P(X), Q{Y))\X ^ Y, with potential function (f> denned as fol- 
lows: 



p 


Q 


<t> 


false 


false 


1 


false 


true 


1 


true 


false 





true 


true 


1 



Note that the parfactor model M', similar to the WMC model M, is a 2-logvar model. □ 

Given any WMC model M, this transformation maps each clause in M to a parfactor that involves 
the same atoms, in the resulting parfactor model M'. As such, each clause is mapped to a parfactor 
with the same (number of) logvars. This transformation thus maps any fc-WFOMC model into an 
equivalent /c-logvar parfactor model. 
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